Member since
07-06-2016
1
Post
0
Kudos Received
0
Solutions
07-06-2016
09:33 AM
I don't understand to config HA-mode when i use hdfsbolt. You can give me some inform to load hdfs-site.xml or core-site.xml or other way to understand HA mode. HdfsBolt bolt = new HdfsBolt()
.withFsUrl("hdfs://ha-cluster")
.withFileNameFormat(fileNameFormat) .withRecordFormat(format)
.withRotationPolicy(rotationPolicy)
.withSyncPolicy(syncPolicy); When i use .withFsUrl("hdfs://pn2:9000") my topology understand but i used .withFsUrl("hdfs://ha-cluster") is not work. java.lang.RuntimeException: Error preparing HdfsBolt: java.net.UnknownHostException: ha-cluster
at org.apache.storm.hdfs.bolt.AbstractHdfsBolt.prepare(AbstractHdfsBolt.java:109) ~[stormjar.jar:na]
at backtype.storm.daemon.executor$fn__3439$fn__3451.invoke(executor.clj:699) ~[storm-core-0.9.6.jar:0.9.6]
at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461) ~[storm-core-0.9.6.jar:0.9.6]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
Caused by: java.lang.IllegalArgumentException: java.net.UnknownHostException: ha-cluster
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:378) ~[stormjar.jar:na]
at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310) ~[stormjar.jar:na]
at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176) ~[stormjar.jar:na]
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678) ~[stormjar.jar:na]
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619) ~[stormjar.jar:na]
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149) ~[stormjar.jar:na]
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653) ~[stormjar.jar:na]
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92) ~[stormjar.jar:na]
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687) ~[stormjar.jar:na]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669) ~[stormjar.jar:na]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) ~[stormjar.jar:na]
at org.apache.storm.hdfs.bolt.HdfsBolt.doPrepare(HdfsBolt.java:86) ~[stormjar.jar:na]
at org.apache.storm.hdfs.bolt.AbstractHdfsBolt.prepare(AbstractHdfsBolt.java:105) ~[stormjar.jar:na]
... 4 common frames omitted
... View more