Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to load hive-site,hdfs-site,core-site xml through command prompt/keep it separately without placing under src/main/resources folder but to load while submitting storm jar?

How to load hive-site,hdfs-site,core-site xml through command prompt/keep it separately without placing under src/main/resources folder but to load while submitting storm jar?

New Contributor

I have storm cluster which is non kerberized connecting to a hdfs cluster which is kerberized and want configuration files(

core-site.xml, hdfs-site.xml, hive-site.xml) to be added to the topology jar, which need to be overridden for the particular storm topology. I should be in position to package the config files per target runtime environment without re-building the storm topology jar.

Note:

ambari 2.6.3 and storm 1.1.0

1 REPLY 1

Re: How to load hive-site,hdfs-site,core-site xml through command prompt/keep it separately without placing under src/main/resources folder but to load while submitting storm jar?

New Contributor

Followed the below steps

1.Place the keytab file that is generated for the storm user in all Non kerberzied nodes(/etc/security/keytabs/)

2.Place the krb5.conf(obtained from Kerberized cluster) in all the Non kerberzied nodes(/etc)

3.Add the below code in the class that is extending HDFSBolt as config parameters

map.put(HdfsSecurityUtil.STORM_KEYTAB_FILE_KEY, “/etc/security/keytabs/storm.headless.keytabs”); map.put(HdfsSecurityUtil.STORM_USER_NAME_KEY, “storm@xyz.com”);

4.Repackage the storm topology jar using jar utility with the necessary files such as hdfs-site.xml,core-site.xml,hive-site.xml. The jar command will be jar -uvf <storm-jar> hdfs-site.xml core-site.xml hive-site.xml

Don't have an account?
Coming from Hortonworks? Activate your account here