Reply
Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Can't read data from HBase using pig and HBaseStorageHandler

Hi, I'm trying to use HBaseStorage to read data from HBase
1. I do persist smth to hbase each day using hbase-client java api
2. using HBaseStorage vis oozie
Now I failed to read persisted data using pig script via HUE or plain pig.
I don't have any problem reading data using java client api.
What do I do wrong?
 
Caused by: java.lang.NumberFormatException: For input string: "4f8:0:a0a1::add:1010"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:492)
at java.lang.Integer.parseInt(Integer.java:527)
at com.sun.jndi.dns.DnsClient.<init>(DnsClient.java:125)
at com.sun.jndi.dns.Resolver.<init>(Resolver.java:61)
at com.sun.jndi.dns.DnsContext.getResolver(DnsContext.java:570)
at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:430)
at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:231)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:139)
at com.sun.jndi.toolkit.url.GenericURLDirContext.getAttributes(GenericURLDirContext.java:103)
at javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:142)
at org.apache.hadoop.net.DNS.reverseDns(DNS.java:84)
at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:228)
at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:191)
at org.apache.pig.backend.hadoop.hbase.HBaseTableInputFormat.getSplits(HBaseTableInputFormat.java:87)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
... 18 more
Cloudera Employee
Posts: 88
Registered: ‎01-08-2014

Re: Can't read data from HBase using pig and HBaseStorageHandler

Your machine is configured for IPv6. IPv6 is not supported, and various components will have problems with it. You should ensure all nodes are only using IPv4.

Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Re: Can't read data from HBase using pig and HBaseStorageHandler

I know... I have limited access to OS, how can I check that ipv6 is totally disabled?

Cloudera Employee
Posts: 88
Registered: ‎01-08-2014

Re: Can't read data from HBase using pig and HBaseStorageHandler

It depends on your operating system. You should look up network configuration docs specific to the operating system you're using.

 

You can usually check lsmod to see if ipv6 shows up, or look at the output of ifconfig to see if adapters have inet6 addresses.

Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Re: Can't read data from HBase using pig and HBaseStorageHandler

Hi, it was datacenter dns ipv6

So we just deleted ALL packages having ipv6 in their names.

Strange, we met it problem first time for the last half year. We met it trying to use pig HBaseStorageHandler to read HBase data.

We were using it for writing before and there were no poblems with it.

Cloudera Employee
Posts: 88
Registered: ‎01-08-2014

Re: Can't read data from HBase using pig and HBaseStorageHandler

Many of the components have gotten better about ignoring IPv6 when it's present so long as IPv4 works. Best practice is still to keep it removed because you'll eventually hit edge cases like this.

Announcements