Connection reset by peer when query specific table

[#1]

Server version: impalad version 2.6.0-cdh5.8.4 RELEASE (build 207450616f75adbe082a4c2e1145a2384da83fa6) .

I encounter a error when I query specific table using impala shell. The immediate msg is : "Socket error 104: Connection reset by peer" . When I do SQL query like "select * from table1" via HUE there is a error saying :"Could not connect to cloudera2:21050 (code THRIFTTRANSPORT): TTransportException('Could not connect to cloudera2:21050',)". but it is ok to do "show database", "select" from other tables.

I check the log file impalad.INFO via http://[impala-host]:25000/logs. There is no specially signs to indicate that failure. Is there any other log file I should check? What should I do for debugging.

Help !

Davey

[#2]

Server version: impalad version 2.6.0-cdh5.8.4 RELEASE (build 207450616f75adbe082a4c2e1145a2384da83fa6) I encounter a error when I query specific table using impala shell. The immediate msg is : "Socket error 104: Connection reset by peer" . When I do SQL query via HUE there is a error saying :"Could not connect to cloudera2:21050 (code THRIFTTRANSPORT): TTransportException('Could not connect to cloudera2:21050',)". But it is ok to do "show database", "select" from other tables. I check the log file impalad.INFO via http://[impala-host]:25000/logs. There is no specially signs to indicate that failure. Is there any other log file I should check? What should I do for debugging. Help ! Davey

[#3]

I'm sorry to make duplicated posts and the format is a mess.

I try again using the same select query with limit 10. The connecion is not reset this time but print another error msg: Application Exception : locale::facet::_S_create_c_locale name not valid.

and I found some weird logs in impalad.INFO :

I0314 14:49:13.891793 175230 Frontend.java:875] analyze query select * from staging limit 10
I0314 14:49:14.029599 175230 Frontend.java:953] create plan
I0314 14:49:14.133545 175230 HdfsScanNode.java:363] collecting partitions for table staging
I0314 14:49:14.133721 175230 HdfsScanNode.java:415] computeStats HdfsScan: cardinality_=-1
I0314 14:49:14.133883 175230 HdfsScanNode.java:471] computeNumNodes totalRanges=2 localRanges=1 remoteRanges=0 localHostSet.size=1 clusterNodes=1
I0314 14:49:14.133975 175230 HdfsScanNode.java:418] computeStats HdfsScan: #nodes=1
I0314 14:49:14.134367 175230 HdfsScanNode.java:363] collecting partitions for table staging
I0314 14:49:14.137238 175230 HdfsScanNode.java:415] computeStats HdfsScan: cardinality_=10
I0314 14:49:14.137370 175230 HdfsScanNode.java:471] computeNumNodes totalRanges=2 localRanges=1 remoteRanges=0 localHostSet.size=1 clusterNodes=1
I0314 14:49:14.137439 175230 HdfsScanNode.java:418] computeStats HdfsScan: #nodes=1
I0314 14:49:14.147675 175230 Planner.java:128] desctbl: tuples:
TupleDescriptor{id=0, name=BaseTableRef dns.staging, tbl=dns.staging, byte_size=432, is_materialized=true, slots=[SlotDescriptor{id=0, path=dns.staging.id, type=INT, materialized=true, byteSize=4, byteOffset=24, nullable=true, nullIndicatorByte=1, nullIndicatorBit=4, slotIdx=12, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=1, path=dns.staging.unixtime, type=BIGINT, materialized=true, byteSize=8, byteOffset=136, nullable=true, nullIndicatorByte=4, nullIndicatorBit=7, slotIdx=39, stats=ColumnStats{avgSerializedSize_=8.0, maxSize_=8, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=2, path=dns.staging.time, type=BIGINT, materialized=true, byteSize=8, byteOffset=144, nullable=true, nullIndicatorByte=5, nullIndicatorBit=0, slotIdx=40, stats=ColumnStats{avgSerializedSize_=8.0, maxSize_=8, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=3, path=dns.staging.qname, type=STRING, materialized=true, byteSize=16, byteOffset=160, nullable=true, nullIndicatorByte=5, nullIndicatorBit=2, slotIdx=42, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=4, path=dns.staging.domainname, type=STRING, materialized=true, byteSize=16, byteOffset=176, nullable=true, nullIndicatorByte=5, nullIndicatorBit=3, slotIdx=43, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=5, path=dns.staging.len, type=INT, materialized=true, byteSize=4, byteOffset=28, nullable=true, nullIndicatorByte=1, nullIndicatorBit=5, slotIdx=13, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=6, path=dns.staging.frag, type=INT, materialized=true, byteSize=4, byteOffset=32, nullable=true, nullIndicatorByte=1, nullIndicatorBit=6, slotIdx=14, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=7, path=dns.staging.ttl, type=INT, materialized=true, byteSize=4, byteOffset=36, nullable=true, nullIndicatorByte=1, nullIndicatorBit=7, slotIdx=15, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=8, path=dns.staging.ipv, type=INT, materialized=true, byteSize=4, byteOffset=40, nullable=true, nullIndicatorByte=2, nullIndicatorBit=0, slotIdx=16, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=9, path=dns.staging.prot, type=INT, materialized=true, byteSize=4, byteOffset=44, nullable=true, nullIndicatorByte=2, nullIndicatorBit=1, slotIdx=17, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=10, path=dns.staging.src, type=STRING, materialized=true, byteSize=16, byteOffset=192, nullable=true, nullIndicatorByte=5, nullIndicatorBit=4, slotIdx=44, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=11, path=dns.staging.srcp, type=INT, materialized=true, byteSize=4, byteOffset=48, nullable=true, nullIndicatorByte=2, nullIndicatorBit=2, slotIdx=18, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=12, path=dns.staging.dst, type=STRING, materialized=true, byteSize=16, byteOffset=208, nullable=true, nullIndicatorByte=5, nullIndicatorBit=5, slotIdx=45, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=13, path=dns.staging.dstp, type=INT, materialized=true, byteSize=4, byteOffset=52, nullable=true, nullIndicatorByte=2, nullIndicatorBit=3, slotIdx=19, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=14, path=dns.staging.udp_sum, type=INT, materialized=true, byteSize=4, byteOffset=56, nullable=true, nullIndicatorByte=2, nullIndicatorBit=4, slotIdx=20, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=15, path=dns.staging.dns_len, type=INT, materialized=true, byteSize=4, byteOffset=60, nullable=true, nullIndicatorByte=2, nullIndicatorBit=5, slotIdx=21, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=16, path=dns.staging.aa, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=8, nullable=true, nullIndicatorByte=0, nullIndicatorBit=0, slotIdx=0, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=17, path=dns.staging.tc, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=9, nullable=true, nullIndicatorByte=0, nullIndicatorBit=1, slotIdx=1, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=18, path=dns.staging.rd, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=10, nullable=true, nullIndicatorByte=0, nullIndicatorBit=2, slotIdx=2, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=19, path=dns.staging.ra, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=11, nullable=true, nullIndicatorByte=0, nullIndicatorBit=3, slotIdx=3, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=20, path=dns.staging.z, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=12, nullable=true, nullIndicatorByte=0, nullIndicatorBit=4, slotIdx=4, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=21, path=dns.staging.ad, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=13, nullable=true, nullIndicatorByte=0, nullIndicatorBit=5, slotIdx=5, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=22, path=dns.staging.cd, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=14, nullable=true, nullIndicatorByte=0, nullIndicatorBit=6, slotIdx=6, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=23, path=dns.staging.ancount, type=INT, materialized=true, byteSize=4, byteOffset=64, nullable=true, nullIndicatorByte=2, nullIndicatorBit=6, slotIdx=22, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=24, path=dns.staging.arcount, type=INT, materialized=true, byteSize=4, byteOffset=68, nullable=true, nullIndicatorByte=2, nullIndicatorBit=7, slotIdx=23, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=25, path=dns.staging.nscount, type=INT, materialized=true, byteSize=4, byteOffset=72, nullable=true, nullIndicatorByte=3, nullIndicatorBit=0, slotIdx=24, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=26, path=dns.staging.qdcount, type=INT, materialized=true, byteSize=4, byteOffset=76, nullable=true, nullIndicatorByte=3, nullIndicatorBit=1, slotIdx=25, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=27, path=dns.staging.opcode, type=INT, materialized=true, byteSize=4, byteOffset=80, nullable=true, nullIndicatorByte=3, nullIndicatorBit=2, slotIdx=26, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=28, path=dns.staging.rcode, type=INT, materialized=true, byteSize=4, byteOffset=84, nullable=true, nullIndicatorByte=3, nullIndicatorBit=3, slotIdx=27, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=29, path=dns.staging.qtype, type=INT, materialized=true, byteSize=4, byteOffset=88, nullable=true, nullIndicatorByte=3, nullIndicatorBit=4, slotIdx=28, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=30, path=dns.staging.qclass, type=INT, materialized=true, byteSize=4, byteOffset=92, nullable=true, nullIndicatorByte=3, nullIndicatorBit=5, slotIdx=29, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=31, path=dns.staging.country, type=STRING, materialized=true, byteSize=16, byteOffset=224, nullable=true, nullIndicatorByte=5, nullIndicatorBit=6, slotIdx=46, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=32, path=dns.staging.asn, type=STRING, materialized=true, byteSize=16, byteOffset=240, nullable=true, nullIndicatorByte=5, nullIndicatorBit=7, slotIdx=47, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=33, path=dns.staging.edns_udp, type=INT, materialized=true, byteSize=4, byteOffset=96, nullable=true, nullIndicatorByte=3, nullIndicatorBit=6, slotIdx=30, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=34, path=dns.staging.edns_version, type=SMALLINT, materialized=true, byteSize=2, byteOffset=20, nullable=true, nullIndicatorByte=1, nullIndicatorBit=3, slotIdx=11, stats=ColumnStats{avgSerializedSize_=2.0, maxSize_=2, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=35, path=dns.staging.edns_do, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=15, nullable=true, nullIndicatorByte=0, nullIndicatorBit=7, slotIdx=7, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=36, path=dns.staging.edns_ping, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=16, nullable=true, nullIndicatorByte=1, nullIndicatorBit=0, slotIdx=8, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=37, path=dns.staging.edns_nsid, type=STRING, materialized=true, byteSize=16, byteOffset=256, nullable=true, nullIndicatorByte=6, nullIndicatorBit=0, slotIdx=48, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=38, path=dns.staging.edns_dnssec_dau, type=STRING, materialized=true, byteSize=16, byteOffset=272, nullable=true, nullIndicatorByte=6, nullIndicatorBit=1, slotIdx=49, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=39, path=dns.staging.edns_dnssec_dhu, type=STRING, materialized=true, byteSize=16, byteOffset=288, nullable=true, nullIndicatorByte=6, nullIndicatorBit=2, slotIdx=50, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=40, path=dns.staging.edns_dnssec_n3u, type=STRING, materialized=true, byteSize=16, byteOffset=304, nullable=true, nullIndicatorByte=6, nullIndicatorBit=3, slotIdx=51, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=41, path=dns.staging.edns_client_subnet, type=STRING, materialized=true, byteSize=16, byteOffset=320, nullable=true, nullIndicatorByte=6, nullIndicatorBit=4, slotIdx=52, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=42, path=dns.staging.edns_other, type=STRING, materialized=true, byteSize=16, byteOffset=336, nullable=true, nullIndicatorByte=6, nullIndicatorBit=5, slotIdx=53, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=43, path=dns.staging.edns_client_subnet_asn, type=STRING, materialized=true, byteSize=16, byteOffset=352, nullable=true, nullIndicatorByte=6, nullIndicatorBit=6, slotIdx=54, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=44, path=dns.staging.edns_client_subnet_country, type=STRING, materialized=true, byteSize=16, byteOffset=368, nullable=true, nullIndicatorByte=6, nullIndicatorBit=7, slotIdx=55, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=45, path=dns.staging.labels, type=INT, materialized=true, byteSize=4, byteOffset=100, nullable=true, nullIndicatorByte=3, nullIndicatorBit=7, slotIdx=31, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=46, path=dns.staging.res_len, type=INT, materialized=true, byteSize=4, byteOffset=104, nullable=true, nullIndicatorByte=4, nullIndicatorBit=0, slotIdx=32, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=47, path=dns.staging.svr, type=STRING, materialized=true, byteSize=16, byteOffset=384, nullable=true, nullIndicatorByte=7, nullIndicatorBit=0, slotIdx=56, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=48, path=dns.staging.time_micro, type=BIGINT, materialized=true, byteSize=8, byteOffset=152, nullable=true, nullIndicatorByte=5, nullIndicatorBit=1, slotIdx=41, stats=ColumnStats{avgSerializedSize_=8.0, maxSize_=8, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=49, path=dns.staging.resp_frag, type=INT, materialized=true, byteSize=4, byteOffset=108, nullable=true, nullIndicatorByte=4, nullIndicatorBit=1, slotIdx=33, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=50, path=dns.staging.proc_time, type=INT, materialized=true, byteSize=4, byteOffset=112, nullable=true, nullIndicatorByte=4, nullIndicatorBit=2, slotIdx=34, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=51, path=dns.staging.is_google, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=17, nullable=true, nullIndicatorByte=1, nullIndicatorBit=1, slotIdx=9, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=52, path=dns.staging.is_opendns, type=BOOLEAN, materialized=true, byteSize=1, byteOffset=18, nullable=true, nullIndicatorByte=1, nullIndicatorBit=2, slotIdx=10, stats=ColumnStats{avgSerializedSize_=1.0, maxSize_=1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=53, path=dns.staging.dns_res_len, type=INT, materialized=true, byteSize=4, byteOffset=116, nullable=true, nullIndicatorByte=4, nullIndicatorBit=3, slotIdx=35, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=54, path=dns.staging.server_location, type=STRING, materialized=true, byteSize=16, byteOffset=400, nullable=true, nullIndicatorByte=7, nullIndicatorBit=1, slotIdx=57, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=-1, numNulls_=-1}}, SlotDescriptor{id=55, path=dns.staging.year, type=INT, materialized=true, byteSize=4, byteOffset=120, nullable=true, nullIndicatorByte=4, nullIndicatorBit=4, slotIdx=36, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=1, numNulls_=0}}, SlotDescriptor{id=56, path=dns.staging.month, type=INT, materialized=true, byteSize=4, byteOffset=124, nullable=true, nullIndicatorByte=4, nullIndicatorBit=5, slotIdx=37, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=1, numNulls_=0}}, SlotDescriptor{id=57, path=dns.staging.day, type=INT, materialized=true, byteSize=4, byteOffset=128, nullable=true, nullIndicatorByte=4, nullIndicatorBit=6, slotIdx=38, stats=ColumnStats{avgSerializedSize_=4.0, maxSize_=4, numDistinct_=2, numNulls_=0}}, SlotDescriptor{id=58, path=dns.staging.server, type=STRING, materialized=true, byteSize=16, byteOffset=416, nullable=true, nullIndicatorByte=7, nullIndicatorBit=2, slotIdx=58, stats=ColumnStats{avgSerializedSize_=15.0, maxSize_=-1, numDistinct_=1, numNulls_=0}}]}
I0314 14:49:14.150130 175230 Planner.java:129] resultexprs: SlotRef{path=id, type=INT, id=0} SlotRef{path=unixtime, type=BIGINT, id=1} SlotRef{path=time, type=BIGINT, id=2} SlotRef{path=qname, type=STRING, id=3} SlotRef{path=domainname, type=STRING, id=4} SlotRef{path=len, type=INT, id=5} SlotRef{path=frag, type=INT, id=6} SlotRef{path=ttl, type=INT, id=7} SlotRef{path=ipv, type=INT, id=8} SlotRef{path=prot, type=INT, id=9} SlotRef{path=src, type=STRING, id=10} SlotRef{path=srcp, type=INT, id=11} SlotRef{path=dst, type=STRING, id=12} SlotRef{path=dstp, type=INT, id=13} SlotRef{path=udp_sum, type=INT, id=14} SlotRef{path=dns_len, type=INT, id=15} SlotRef{path=aa, type=BOOLEAN, id=16} SlotRef{path=tc, type=BOOLEAN, id=17} SlotRef{path=rd, type=BOOLEAN, id=18} SlotRef{path=ra, type=BOOLEAN, id=19} SlotRef{path=z, type=BOOLEAN, id=20} SlotRef{path=ad, type=BOOLEAN, id=21} SlotRef{path=cd, type=BOOLEAN, id=22} SlotRef{path=ancount, type=INT, id=23} SlotRef{path=arcount, type=INT, id=24} SlotRef{path=nscount, type=INT, id=25} SlotRef{path=qdcount, type=INT, id=26} SlotRef{path=opcode, type=INT, id=27} SlotRef{path=rcode, type=INT, id=28} SlotRef{path=qtype, type=INT, id=29} SlotRef{path=qclass, type=INT, id=30} SlotRef{path=country, type=STRING, id=31} SlotRef{path=asn, type=STRING, id=32} SlotRef{path=edns_udp, type=INT, id=33} SlotRef{path=edns_version, type=SMALLINT, id=34} SlotRef{path=edns_do, type=BOOLEAN, id=35} SlotRef{path=edns_ping, type=BOOLEAN, id=36} SlotRef{path=edns_nsid, type=STRING, id=37} SlotRef{path=edns_dnssec_dau, type=STRING, id=38} SlotRef{path=edns_dnssec_dhu, type=STRING, id=39} SlotRef{path=edns_dnssec_n3u, type=STRING, id=40} SlotRef{path=edns_client_subnet, type=STRING, id=41} SlotRef{path=edns_other, type=STRING, id=42} SlotRef{path=edns_client_subnet_asn, type=STRING, id=43} SlotRef{path=edns_client_subnet_country, type=STRING, id=44} SlotRef{path=labels, type=INT, id=45} SlotRef{path=res_len, type=INT, id=46} SlotRef{path=svr, type=STRING, id=47} SlotRef{path=time_micro, type=BIGINT, id=48} SlotRef{path=resp_frag, type=INT, id=49} SlotRef{path=proc_time, type=INT, id=50} SlotRef{path=is_google, type=BOOLEAN, id=51} SlotRef{path=is_opendns, type=BOOLEAN, id=52} SlotRef{path=dns_res_len, type=INT, id=53} SlotRef{path=server_location, type=STRING, id=54} SlotRef{path=year, type=INT, id=55} SlotRef{path=month, type=INT, id=56} SlotRef{path=day, type=INT, id=57} SlotRef{path=server, type=STRING, id=58}
I0314 14:49:14.150208 175230 Planner.java:130] finalize plan fragments
I0314 14:49:14.172600 175230 Frontend.java:996] get scan range locations
I0314 14:49:14.207159 175230 Planner.java:329] Estimated per-host peak memory requirement: 0
I0314 14:49:14.207227 175230 Planner.java:330] Estimated per-host virtual cores requirement: 0
I0314 14:49:14.259551 175230 Frontend.java:1074] create result set metadata
I0314 14:49:14.261106 175230 JniFrontend.java:152] Estimated Per-Host Requirements: Memory=0B VCores=0
WARNING: The following tables are missing relevant table and/or column statistics.
dns.staging

00:SCAN HDFS [dns.staging]
   partitions=2/2 files=2 size=2.21MB
   table stats: unavailable
   columns missing stats: id, unixtime, time, qname, domainname, len, frag, ttl, ipv, prot, src, srcp, dst, dstp, udp_sum, dns_len, aa, tc, rd, ra, z, ad, cd, ancount, arcount, nscount, qdcount, opcode, rcode, qtype, qclass, country, asn, edns_udp, edns_version, edns_do, edns_ping, edns_nsid, edns_dnssec_dau, edns_dnssec_dhu, edns_dnssec_n3u, edns_client_subnet, edns_other, edns_client_subnet_asn, edns_client_subnet_country, labels, res_len, svr, time_micro, resp_frag, proc_time, is_google, is_opendns, dns_res_len, server_location
   limit: 10
   hosts=1 per-host-mem=unavailable
   tuple-ids=0 row-size=400B cardinality=10
I0314 14:49:14.274642 175230 RequestPoolService.java:274] resolveRequestPool(pool=, user=entrada): resolved_pool=root.entrada, has_access=true
I0314 14:49:14.274952 175230 RequestPoolService.java:372] getPoolConfig(pool=root.entrada): max_mem_resources=-1, max_requests=-1, max_queued=200,  queue_timeout_ms=0, default_query_options=
I0314 14:49:14.275029 175230 admission-controller.cc:436] Schedule for id=774d77ca58c5746a:9e39906bcff25c8c in pool_name=root.entrada cluster_mem_needed=4.00 GB PoolConfig: max_requests=-1 max_queued=200 max_mem=-1.00 B
I0314 14:49:14.275041 175230 admission-controller.cc:441] Stats: agg_num_running=0, agg_num_queued=0, agg_mem_reserved=0,  local_host(local_mem_admitted=0, num_admitted_running=0, num_queued=0, backend_mem_reserved=0)
I0314 14:49:14.275045 175230 admission-controller.cc:447] Admitted query id=774d77ca58c5746a:9e39906bcff25c8c
I0314 14:49:14.275063 175230 coordinator.cc:384] Exec() query_id=774d77ca58c5746a:9e39906bcff25c8c
I0314 14:49:14.275218 175230 plan-fragment-executor.cc:91] Prepare(): query_id=774d77ca58c5746a:9e39906bcff25c8c instance_id=774d77ca58c5746a:9e39906bcff25c8d
I0314 14:49:14.275486 175230 plan-fragment-executor.cc:201] descriptor table for fragment=774d77ca58c5746a:9e39906bcff25c8d
tuples:

When I incease the limit to 100, the connection is reset again...What's going on ?

Davey

[#4]

Before sending sql queries, I found another error when it is detecting and create new partitions for new parquet data. The impala reset the connetion in the first place.

Starting Impala Shell without Kerberos authentication
Socket error 104: Connection reset by peer
Could not execute command: select count(1) from dns.staging where year=2015 and month=10 and day=30 and server="test"

or 

Error connecting: TTransportException, Could not connect to 10.3.2.199:21000
Not connected to Impala, could not execute queries.
[Tue Mar 14 15:55:52 CST 2017] :Adding partition to table dns.staging failed

[#5]

One possible explanation is a crash if there is some problem with the data file.

Are there any hs_err_pid*.log files in /var/log/impalad? Or any *.dmp files?