We are aiming for Dal-M20 but this is not at a high priority at this point. ... View more

Make sure ui kerberos auth to local rules are configured properly. Once principle from AD is used for negotiation with MIT KDC, there need to be a rule that translate it to local account in Storm UI node. Many times those can be copied from core-site.xml. For example: ui.filter.params:"type": "kerberos""kerberos.principal": "HTTP/nimbus.witzend.com""kerberos.keytab": "/vagrant/keytabs/http.keytab""kerberos.name.rules": "RULE:[2:$1@$0]([jt]t@.*EXAMPLE.COM)s/.*/$MAPRED_USER/ RULE:[2:$1@$0]([nd]n@.*EXAMPLE.COM)s/.*/$HDFS_USER/DEFAULT" Note that rules are listed as string without commas. 2. You will need to create mapping for MIT Domain KDC and correlated resource used for the Domain, in this case Storm UI. You will need to execute following commands on Windows workstation from the command line: ksetup /AddKDC $DOMAIN $KDC ksetup /AddHostToRealmMap $hadoop_resource $Domain Note that this adds registry entries in: HKLM\System\CurrentControlSet\Control\Lsa\Kerberos\HostToRealm If you need to troubleshoot the issues you can try accessing Storm UI within the cluster using curl command. For example: curl -i --negotiate -u:anyUser -b ~/cookiejar.txt -c ~/cookiejar.txt http://storm-ui-hostname:8080/api/v1/cluster/summary This will be helpful to see if kerberos UI configs are working. In order to isolate the issue you can use storm service keytabs as well as user principles. Another important thing to check is to make sure that trust is working properly and encryption types match on both KDCs. ... View more

Shouldn't be a memory constraint. Did you try deleting storm-local dir you can find the location storm.local.dir in /etc/storm/conf/storm.yaml. Stop storm services delete this dir and restart the nimbus. ... View more

broker.list for kafka consumers or producers used for bootstrapping. Consumer or Producer makes a request to get TopicMetadata which tells the clients what are the topic partitions and who are leaders for these partitions so that clients can send requests to the leaders. To answer your question brokerList will be shuffled and it will go through each one of the hosts and makes TopicMetadataRequest if its succeeded it will return , if not it will continue to next broker. ... View more

It depends on how the spout is implemented. Lets look at KafkaSpout the failed messages can be for 2 reasons 1. Downstream bolts are failed to process or called collector.fail 2. Downstream bolts are failed to acknowledge the tuple within topology.message.timeout.secs this is 30secs by default. In two cases you can see the spout failed number go up. But kafkaSpout will replay until the messages are acknowledged . If you are using Acking with storm-core topology it guarantees at least once delivery, i.e there might be duplicates but no message loss. ... View more