Member since
07-24-2018
15
Posts
1
Kudos Received
0
Solutions
08-10-2018
05:10 PM
@sunile.manjee Thank you. You posted this just in time. I had this same issue today and following your 2nd option fixed it.
... View more
08-10-2018
10:11 AM
Hi @Akhil S Naik, Ok, I will close this and re-open. The code formatting is not working for me, works when I edit not after posting. I have run the post-upgrade ambari-server task. Thanks again. Paul
... View more
08-10-2018
09:49 AM
Hi @Akhil S Naik , I started again with a fresh cluster, upgraded ambari server and agent to the latest version. After the upgrade when I go back into ambari and try and restart ambari metrics service I now get this error: (I've restarted the ambari-server etc.) With this in the ambari-server.log: 2018-08-10 09:51:57,394 INFO [agent-register-processor-6] HeartBeatHandler:317 - agentOsType = centos7
2018-08-10 09:51:57,404 INFO [agent-register-processor-6] HostImpl:346 - Received host registration, host=[hostname=cs-eu-hdp-3-0-cluster-w1,fqdn=XXX$
, registrationTime=1533894717394, agentVersion=2.7.0.0
2018-08-10 09:51:57,404 INFO [agent-register-processor-6] TopologyManager:643 - TopologyManager.onHostRegistered: Entering
2018-08-10 09:51:57,404 INFO [agent-register-processor-6] TopologyManager:697 - Host XXX.ax.internal.cloudapp.net re-registered, will not b$
2018-08-10 09:51:57,451 ERROR [clientInboundChannel-7] WebSocketAnnotationMethodMessageHandler:548 - Error while processing handler method exception
org.springframework.messaging.converter.MessageConversionException: Could not write JSON: (was java.lang.ClassCastException) (through reference chain: org.springframework.messaging.support$
at org.springframework.messaging.converter.MappingJackson2MessageConverter.convertToInternal(MappingJackson2MessageConverter.java:342)
at org.springframework.messaging.converter.AbstractMessageConverter.toMessage(AbstractMessageConverter.java:194)
at org.springframework.messaging.converter.CompositeMessageConverter.toMessage(CompositeMessageConverter.java:91)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.doConvert(AbstractMessageSendingTemplate.java:171)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.convertAndSend(AbstractMessageSendingTemplate.java:142)
at org.springframework.messaging.simp.SimpMessagingTemplate.convertAndSendToUser(SimpMessagingTemplate.java:225)
at org.springframework.messaging.simp.SimpMessagingTemplate.convertAndSendToUser(SimpMessagingTemplate.java:208)
at org.apache.ambari.server.api.AmbariSendToMethodReturnValueHandler.handleReturnValue(AmbariSendToMethodReturnValueHandler.java:84)
at org.springframework.messaging.handler.invocation.HandlerMethodReturnValueHandlerComposite.handleReturnValue(HandlerMethodReturnValueHandlerComposite.java:103)
at org.springframework.messaging.handler.invocation.AbstractMethodMessageHandler.processHandlerMethodException(AbstractMethodMessageHandler.java:545)
at org.springframework.messaging.handler.invocation.AbstractMethodMessageHandler.handleMatch(AbstractMethodMessageHandler.java:520)
at org.springframework.messaging.simp.annotation.support.SimpAnnotationMethodMessageHandler.handleMatch(SimpAnnotationMethodMessageHandler.java:495)
at org.springframework.messaging.simp.annotation.support.SimpAnnotationMethodMessageHandler.handleMatch(SimpAnnotationMethodMessageHandler.java:87)
at org.springframework.messaging.handler.invocation.AbstractMethodMessageHandler.handleMessageInternal(AbstractMethodMessageHandler.java:463)
at org.springframework.messaging.handler.invocation.AbstractMethodMessageHandler.handleMessage(AbstractMethodMessageHandler.java:401)
at org.springframework.messaging.support.ExecutorSubscribableChannel$SendTask.run(ExecutorSubscribableChannel.java:135)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.ClassCastException) (through reference chain: org.springframework.messaging.support.ErrorMessage["payload"]->$
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:391)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:351)
at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:316)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFieldsUsing(MapSerializer.java:818)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:637)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:2643)
at org.springframework.messaging.converter.MappingJackson2MessageConverter.convertToInternal(MappingJackson2MessageConverter.java:326)
... 18 more
Caused by: java.lang.ClassCastException
... View more
08-09-2018
03:49 PM
Hi @Akhil S Naik, Yes, that looks to be the problem I am seeing. My version no.'s are: Ambari Server: 2.7.0.0-508 Ambari Agent: 2.7.0.0. Paul
... View more
08-09-2018
01:48 PM
Hi @Akhil S Naik , Thanks, I have run ambari-agent restart multiple times. Here is what happens: I run the command, The ambari-agent logs show a problem listening on ping port 8670 saying that another process is using the port, I am assuming the process is another instance of ambari-agent, the PID returns process name of python, I try killing this process using the PID, almost straight away another process starts and takes it's place, The agent then goes on and returns the heartbeatthread exception I mentioned in previous post. It would appear ambari-agent is un-killable/stopable. There must be a supervisor forcing a restart on failure and this is causing multiple instances to run at once and causing the ping port 8670 problem. I think I can fix that by killing the 2nd process but any idea what's causing the heartbeatthread excepion? Here's one loop of the log after I've cleared up the ping port issue: ambari-agent.log. ambariagent.txt Update: Looking at the ambari-server and agent logs it appears that the agent is registering with the server as the logs there showing it is but then there is a heartbeatexception on the agent and a nullpointerexception on the server and both then seem to loop without completing what they started. Here's the repeating error on ambari-server again. 2018-08-09 15:07:11,984 INFO [agent-register-processor-2] HeartBeatHandler:312 - agentOsType = centos7
2018-08-09 15:07:11,991 INFO [agent-register-processor-2] HostImpl:345 - Received host registration, host=[hostname=cs-eu-dhp-3-0-0-cluster-m2,fqdn=cs-eu-dhp-3-0-0-cluster-m2.XXX-NODE-FQDN-XXX,domain=XXX-NODE-FQDN-XXX,architecture=x86_64,processorcount=4,physicalprocessorcount=4,osname=centos,osversion=7.5.1804,osfamily=redhat,memory=28801164,uptime_hours=3,mounts=(available=22377356,mountpoint=/,used=8551792,percent=28%,size=30929148,device=/dev/sda2,type=xfs)(available=195727412,mountpoint=/mnt/resource,used=61468,percent=1%,size=206290920,device=/dev/sdb1,type=ext4)(available=97673124,mountpoint=/hadoopfs/fs1,used=148500,percent=1%,size=103080888,device=/dev/sdc,type=ext4)]
, registrationTime=1533827231984, agentVersion=2.7.0.0
2018-08-09 15:07:11,991 INFO [agent-register-processor-2] TopologyManager:643 - TopologyManager.onHostRegistered: Entering
2018-08-09 15:07:11,991 INFO [agent-register-processor-2] TopologyManager:697 - Host cs-eu-dhp-3-0-0-cluster-m2.XXX-NODE-FQDN-XXX re-registered, will not be added to the available hosts list
2018-08-09 15:07:11,991 INFO [agent-register-processor-2] HeartbeatController:105 -
java.lang.NullPointerException
at org.apache.ambari.server.state.host.HostImpl.calculateHostStatus(HostImpl.java:1259)
at org.apache.ambari.server.state.host.HostImpl.restoreComponentsStatuses(HostImpl.java:1230)
at org.apache.ambari.server.state.host.HostImpl$HostRegistrationReceived.transition(HostImpl.java:365)
at org.apache.ambari.server.state.host.HostImpl$HostRegistrationReceived.transition(HostImpl.java:333)
at org.apache.ambari.server.state.fsm.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:351)
at org.apache.ambari.server.state.fsm.StateMachineFactory.doTransition(StateMachineFactory.java:293)
at org.apache.ambari.server.state.fsm.StateMachineFactory.access$300(StateMachineFactory.java:39)
at org.apache.ambari.server.state.fsm.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:437)
at org.apache.ambari.server.state.host.HostImpl.handleEvent(HostImpl.java:597)
at org.apache.ambari.server.agent.HeartBeatHandler.handleRegistration(HeartBeatHandler.java:345)
at org.apache.ambari.server.agent.stomp.HeartbeatController.lambda$register$0(HeartbeatController.java:100)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
... View more
08-09-2018
01:29 PM
Hi @Akhil S Naik, Thanks again for the help. Here's where I have got to.
Build Fresh 3 node HDP 3.0 cluster, using data science blueprint (spark2, zepplin etc.), Ambari 2.7, Add Druid service to HDP using Amabri wizard, using default settings, installs successfully, Druid starts, Ambari flags that some services require a restart, I try, I get the roleParams error, restart of ambari server resolves this, services still need a restart, Try and restart service again, worker and compute node restart fine, master node does not even try to start restarts, Looking at the ambari-agent log I see errors stating that agent could not register, ambari server log shows it cannot connect to the master node. INFO 2018-08-09 13:27:13,492 __init__.py:49 - Event from server at /user/ (correlation_id=0): {u'status': u'OK', u'exitstatus': 1, u'id': -1}
INFO 2018-08-09 13:27:13,497 HeartbeatThread.py:128 - Registration response received
ERROR 2018-08-09 13:27:13,498 HeartbeatThread.py:104 - Exception in HeartbeatThread. Re-running the registration
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/ambari_agent/HeartbeatThread.py", line 91, in run
self.register()
File "/usr/lib/ambari-agent/lib/ambari_agent/HeartbeatThread.py", line 131, in register
self.handle_registration_response(response)
File "/usr/lib/ambari-agent/lib/ambari_agent/HeartbeatThread.py", line 189, in handle_registration_response
raise Exception(error_message)
Exception: Registration failed
INFO 2018-08-09 13:27:13,499 transport.py:358 - Receiver loop ended
<br> There are other errors re cluster id=2 configurations missing but assume that is related to not being able to register. These errors loop endlessly. The master node also showinf status of UNHEALTHY in cloudbreak.
... View more
08-08-2018
12:33 PM
Hi @Akhil S Naik, Thanks for trying to help. I get no value found when I search in command-82.json for roleParams. I restarted Ambari and also nothing found after. After I restarted Ambari server I have tried restarting the services again and watching the ambari-server.log file I now see that it is looping on a NullPointerException. The restart in Ambari has stalled at 8% "Restart App Timeline Server" on the master node. Worker and compute node have completed restarts. 2018-08-08 12:29:31,329 INFO [agent-register-processor-5] HeartbeatController:105 -
java.lang.NullPointerException
at org.apache.ambari.server.state.host.HostImpl.calculateHostStatus(HostImpl.java:1259)
at org.apache.ambari.server.state.host.HostImpl.restoreComponentsStatuses(HostImpl.java:1230)
at org.apache.ambari.server.state.host.HostImpl$HostRegistrationReceived.transition(HostImpl.java:365)
at org.apache.ambari.server.state.host.HostImpl$HostRegistrationReceived.transition(HostImpl.java:333)
at org.apache.ambari.server.state.fsm.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:351)
at org.apache.ambari.server.state.fsm.StateMachineFactory.doTransition(StateMachineFactory.java:293)
at org.apache.ambari.server.state.fsm.StateMachineFactory.access$300(StateMachineFactory.java:39)
at org.apache.ambari.server.state.fsm.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:437)
at org.apache.ambari.server.state.host.HostImpl.handleEvent(HostImpl.java:597)
at org.apache.ambari.server.agent.HeartBeatHandler.handleRegistration(HeartBeatHandler.java:345)
at org.apache.ambari.server.agent.stomp.HeartbeatController.lambda$register$0(HeartbeatController.java:100)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
... View more
08-08-2018
11:59 AM
Hi, Any idea why I am getting this error when doing the restart after I have installed Druid service via Ambari on fresh HDP 3.0 cluster? The cluster was deployed using Cloudbreak on Azure. Ambari version 2.7.0.0. stderr: /var/lib/ambari-agent/data/errors-82.txt Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/application_timeline_server.py", line 97, in <module>
ApplicationTimelineServer().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 353, in execute
method(env)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 933, in restart
if componentCategory and componentCategory.strip().lower() == 'CLIENT'.lower():
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/config_dictionary.py", line 73, in __getattr__
raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'roleParams' was not found in configurations dictionary! stdout: /var/lib/ambari-agent/data/output-82.txt 2018-08-08 11:35:16,639 - Stack Feature Version Info: Cluster Stack=3.0, Command Stack=None, Command Version=3.0.0.0-1334 -> 3.0.0.0-1334
2018-08-08 11:35:16,654 - Using hadoop conf dir: /usr/hdp/3.0.0.0-1334/hadoop/conf
2018-08-08 11:35:16,797 - Stack Feature Version Info: Cluster Stack=3.0, Command Stack=None, Command Version=3.0.0.0-1334 -> 3.0.0.0-1334
2018-08-08 11:35:16,801 - Using hadoop conf dir: /usr/hdp/3.0.0.0-1334/hadoop/conf
2018-08-08 11:35:16,802 - Group['hdfs'] {}
2018-08-08 11:35:16,803 - Group['hadoop'] {}
2018-08-08 11:35:16,803 - Group['users'] {}
2018-08-08 11:35:16,804 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-08 11:35:16,804 - User['yarn-ats'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-08 11:35:16,805 - User['druid'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-08 11:35:16,806 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-08 11:35:16,807 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-08 11:35:16,808 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2018-08-08 11:35:16,808 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'users'], 'uid': None}
2018-08-08 11:35:16,809 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop'], 'uid': None}
2018-08-08 11:35:16,810 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-08 11:35:16,811 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2018-08-08 11:35:16,811 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2018-08-08 11:35:16,813 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2018-08-08 11:35:16,817 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if
2018-08-08 11:35:16,818 - Group['hdfs'] {}
2018-08-08 11:35:16,818 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', 'hadoop', u'hdfs']}
2018-08-08 11:35:16,819 - FS Type:
2018-08-08 11:35:16,819 - Directory['/etc/hadoop'] {'mode': 0755}
2018-08-08 11:35:16,832 - File['/usr/hdp/3.0.0.0-1334/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2018-08-08 11:35:16,833 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2018-08-08 11:35:16,846 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2018-08-08 11:35:16,853 - Skipping Execute[('setenforce', '0')] due to not_if
2018-08-08 11:35:16,853 - Directory['/var/log/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'hadoop', 'mode': 0775, 'cd_access': 'a'}
2018-08-08 11:35:16,855 - Directory['/var/run/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'root', 'cd_access': 'a'}
2018-08-08 11:35:16,855 - Changing owner for /var/run/hadoop from 1011 to root
2018-08-08 11:35:16,855 - Changing group for /var/run/hadoop from 988 to root
2018-08-08 11:35:16,855 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'create_parents': True, 'cd_access': 'a'}
2018-08-08 11:35:16,859 - File['/usr/hdp/3.0.0.0-1334/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2018-08-08 11:35:16,860 - File['/usr/hdp/3.0.0.0-1334/hadoop/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2018-08-08 11:35:16,866 - File['/usr/hdp/3.0.0.0-1334/hadoop/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2018-08-08 11:35:16,875 - File['/usr/hdp/3.0.0.0-1334/hadoop/conf/hadoop-metrics2.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2018-08-08 11:35:16,875 - File['/usr/hdp/3.0.0.0-1334/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2018-08-08 11:35:16,876 - File['/usr/hdp/3.0.0.0-1334/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2018-08-08 11:35:16,880 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop', 'mode': 0644}
2018-08-08 11:35:16,884 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2018-08-08 11:35:16,887 - Skipping unlimited key JCE policy check and setup since the Java VM is not managed by Ambari
Command failed after 1 tries
... View more
Labels:
07-27-2018
08:09 AM
@Jay Kumar SenSharma thanks for that information, that is disappointing. So from the looks of this if I am running HDP 3.0 which I think requires Ambari 2.7 (correct me if that is wrong) then there is currently no way for me to add an HDF MPack to HDP 3.0 until HDF 3.2 is released? Do you think this is correct? When they say "in some more days" do you know is that this week, next week, next month, how many days is "some more"? Thanks, Paul
... View more
07-26-2018
08:50 PM
1 Kudo
Hi, I am trying to add the HDF 3.1.2 MPack to my HDP 3.0.0 cluster so that I can use NiFi (along with some HDP service). No matter which way I try and add it I cannot get it to appear in the stack and versions or Services sections of Ambari. I have tried adding the MPack via SSH to the on the master node (installed successfully), also on a fresh cluster I have tried registering the MPack in Cloudbreak and specifying that it be installed during the cluster deployment. Each time it states the MPack has been installed successfully but it is nowhere to be found in Ambari. I cannot find any section to add the base HDF URL as one document says to do nor can I find it to try and register a new version. Only the HDP 3.0 version is shown. Does anyone have a suggestion as to where I'm going wrong? Thanks, Paul
... View more
07-26-2018
03:39 PM
Thanks dsun, that has worked for me. Note I used Dominika's blueprint below but don't know if it makes any difference.
... View more
07-26-2018
03:38 PM
Thanks, Dominika, I used your Blueprint and it didn't seem to need me to add the repo's, it had the correct information there automatically.
... View more
07-24-2018
07:04 PM
Hi, I just got a Cloudbreak 2.7.1 deployer server running. The press release for HDP 3.0 mentioned that is was possible to deploy 3.0 using Cloudbreak but HDP 3.0 isn't an available option under 2.7.1 (HDP 2.6 only). Do I need to upgrade Cloudbreak, add an MPack or how should I deploy HDP 3.0 via Cloudbreak? Paul
... View more
07-24-2018
06:42 PM
Thanks, @Dominika Bialek.
That has worked, I got an error initially as the default Standard_DS2
VM size isn't available in Western Europe, changing this to
Standard_DS2_v2 did the trick.
... View more
07-24-2018
02:00 PM
Hi, Is anyone experiencing a problem trying to deploy Cloudbreak deployer from the Azure Marketplace? I start the deployment and the resource group and resources are created but then the cbdeployerVM/installcustomscript runs forever eventually failing on a timeout error. See screenshot. I cannot get this to work, I've tried using different regions (North & West Europe) but no difference.
... View more
Labels:
