Created on 01-16-2015 09:35 AM - edited 09-16-2022 02:19 AM
I have installed HBase 0.94.15-cdh4.7.0 and having problem with creating a table from hbase shell. Hbase successfully creates two tables, but fails to create the third (hangs up). Here is my log from hbase:
2014-12-06 18:00:22,413 INFO org.apache.hadoop.hbase.master.handler.CreateTableHandler: Attempting to create the table UserGroups 2014-12-06 18:00:22,431 INFO org.apache.hadoop.hdfs.DFSClient: Exception in createBlockOutputStream java.net.NoRouteToHostException: No route to host at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:207) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:528) at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1305) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1131) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1091) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:502) 2014-12-06 18:00:22,431 INFO org.apache.hadoop.hdfs.DFSClient: Abandoning BP-1981556914-172.17.100.181-1406029837178:blk_5061750426017945617_24524723 2014-12-06 18:00:22,432 INFO org.apache.hadoop.hdfs.DFSClient: Excluding datanode 172.17.100.183:50010 2014-12-06 18:00:22,453 INFO org.apache.hadoop.hbase.regionserver.HRegion: creating HRegion UserGroups HTD == 'UserGroups', {NAME => 'Group', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'} RootDir = hdfs://bman.dc2:8020/hbase/.tmp Table name == UserGroups 2014-12-06 18:00:22,472 INFO org.apache.hadoop.hdfs.DFSClient: Exception in createBlockOutputStream java.net.NoRouteToHostException: No route to host at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:207) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:528) at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1305) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1131) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1091) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:502) 2014-12-06 18:00:22,473 INFO org.apache.hadoop.hdfs.DFSClient: Abandoning BP-1981556914-172.17.100.181-1406029837178:blk_-6499242959049007209_24524726 2014-12-06 18:00:22,474 INFO org.apache.hadoop.hdfs.DFSClient: Excluding datanode 172.17.100.183:50010 2014-12-06 18:00:22,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor config now ... 2014-12-06 18:00:22,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed UserGroups,,1417874422380.08f53c8488588eb88dd14e9613765937. 2014-12-06 18:00:22,516 INFO org.apache.hadoop.hbase.catalog.MetaEditor: Added 1 regions in META 2014-12-06 18:00:22,516 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1 region(s) round-robin across 3 server(s) 2014-12-06 18:00:22,531 INFO org.apache.hadoop.hbase.master.AssignmentManager: bh1.dc2,60020,1417873861342 outstanding calls=1 of total=1 2014-12-06 18:00:23,518 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
It seems that problem is caused not by the creation of table, but by some general network error. What may cause this problem?
Created 01-17-2015 03:48 PM
Created 01-24-2015 01:51 PM
Hello Gautam, thanks for reply.
The problem was in iptables at one of the data nodes.
It's a bit strange, because previous tables were created successfully, but turning iptables off solved the problem.