Short Description: This article walks you through how to deploy a custom parser on kerberized cluster in Metron . Article When adding a net new data source to Metron, the first step is to create kafka topic 'sensor_topic' to push the events from the new telemetry data source into Metron. The second step is to configure Metron to parse the telemetry data source so that downstream processing can be done on it. This wiki doc will walk you through how to perform both of these steps. In this article I have added steps to perform the same operation on a kerberised cluster. On kerberised cluster we need to add the required ACL's to metron user and parser group for the created topic. Also we need to provide the storm authorization while deploying the topology.
1. Switch to the kafka user .
2. create sensor topic
/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic sensor_topic_name --partitions 1 --replication-factor 1
3. Add required ACL to metron user /usr/hdp/current/kafka-broker/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=nat-r7-hets-metron-1:2181 --add --allow-principal User:metron --operation All --topic 'sensor_topic_name' --cluster
4. Add required ACL to group /usr/hdp/current/kafka-broker/bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=$ZOOKEEPER --add --allow-principal User:metron --group sensor_topic_name_parser
5. Switch to the metron user and acquire a Kerberos ticket for the metron principal. su metron
kinit -kt /etc/security/keytabs/metron.headless.keytab metron@EXAMPLE.COM
6. Deploy the new Parser Topology /usr/hcp/188.8.131.52-1/metron/bin/start_parser_topology.sh -k $KAFKA_BROKER -z $ZOOKEEPER -s sensor_name -ksp PLAINTEXTSASL -e ~metron/.storm/storm.config
... View more
@Wang Ao since you used the metron-rest deb to install , You can try manually creating those users, refer below docs https://docs.hortonworks.com/HDPDocuments/HCP1/HCP-1.4.1/bk_installation/content/installing_rest_app_manually.html
... View more
I found the solution ... The problem was, users were created before I configuring the KDC to issue renewable tickets, I was under the impression that setting the max_life and max_renewable_life in /var/kerberos/krb5kdc/kdc.conf and restarting the kadmin and krb5kdc services would be enough, but as the values were already stored in KDC it didn’t work. So, as a quick fix I set the renew lifetime for the existing user and krbtgt realm. I think I need to recreate the KDB using "kdb5_util create -s" as even for the new users I see the max_renewable_life is set to 0. below are the commands to set the renew life time for the eisting users Modify the appropriate principals to allow renewable tickets using the following commands. Adjust the parameters to match your desired KDC parameters: kadmin.local -q "modprinc -maxlife 1days -maxrenewlife 7days +allow_renewable krbtgt/EXAMPLE.COM@EXAMPLE.COM"
kadmin.local -q "modprinc -maxlife 1days -maxrenewlife 7days +allow_renewable metron@EXAMPLE.COM"
... View more