@Michael Bronson HWX doesn't recommend upgrading an individual HDP component because one never knows the incompatibilities that could impact the other components and component selective upgrades tend to be a nightmare during a version upgrade The lastest HDP Kafka version is 11-2.1.x delivered by HDP 3.1 but ASF has its own rollout version and naming convention HTH ... View more

@Michael Bronson As we see that it is basically a "500 Error" which basically indicates an Internal Server Error hence you must see a very detailed Stack Trace inside your ambari-server.log. Can you please share the complete ambari-server.log so that we can check what might be failing. ... View more

@Michael Bronson No, unfortunately, I don't have a test cluster, the configuration looks straight forward just create a yaml i.e kafka.yaml file in /etc/kafka_discovery which you export as KAFKA_DISCOVERY_DIR look at the README.md file. Can you tokenize your sensitive hostname and share the YAML file you created? I am sure we can sort that out I can only spin a single node kafka broker this weekend and test. Please revert ... View more

@Michael Bronson Zookeeper's usually have the same port so I suggest you have a look at the brokers. Please have a look at my response to a similar problem https://community.hortonworks.com/questions/235766/kafka-socket-server-failed-bind-exception.html?childToView=234784#answer-234784 HTH ... View more

@Michael Bronson Step1). You can get all the Hostnames where the DataNode is present using the following API call: # curl --user admin:admin -H 'X-Requested-By: ambari' -X GET "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/services/HDFS/components/DATANODE?fields=host_components/HostRoles/host_name" | grep host_name | awk '{print $NF}' | awk -F'"' '{print $2}' . Step2). Once you have the list of the hosts where the Datanode is installed (using above API call) then you can use it to make the following API call using some shell script to replace the $HOST with the hostname. # curl --user admin:admin -H 'X-Requested-By: ambari' -X POST -d '{"RequestInfo":{"command":"RESTART","context":"Restart all components on $HOST","operation_level":{"level":"HOST","cluster_name":"NewCluster"}},"Requests/resource_filters":[{"service_name":"HDFS","component_name":"DATANODE","hosts":"$HOST"}, {"service_name":"YARN","component_name":"NODEMANAGER","hosts":"$HOST"}, {"service_name":"AMBARI_METRICS","component_name":"METRICS_MONITOR","hosts":"$HOST"} ]}' http://ambariserver.example.com:8080/api/v1/clusters/NewCluster/requests . NOTE: In the above call please make sure to replace the $HOST with the hostname one by one (using some shell script Loop iteration) which we retrieved from the previous API call. Also please replace the "NewCluster" with your own cluster name. Also please replace "ambariserver.example.com" with your ambari hostname and credentials accordingly. ... View more