Yep it was exactly this. We had a bunch of impala-shell commands with the -r argument, thus we were invalidating metadata on many parallel processes. We've removed invalidate metadata and refresh statements in a lot of places based on the fact that it's not needed for much of our Impala ETL processes. Has any thought been put into somehow registering these metadata refreshes in the statestore so that if similar requests are running they don't overwhelm the metastore? Either that or post a warning when there are too many metastore refreshes running at the same time? We spent a lot of time digging in on this so anything to help others who encounter similar issues would probably be a good thing. Thanks a lot for your reply, Alex.
... View more
Experiencing frequent errors during table inserts using Impala shell via Oozie. We are on CDH 5.3.1 and running Impala behind a load balancer. Starting Impala Shell without Kerberos authentication
Connected to <removed>
Server version: impalad version 2.1.1-cdh5 RELEASE (build 7901877736e29716147c4804b0841afc4ebc9037) Error communicating with impalad: TSocket read 0 bytes
Could not execute command: insert into table Originally I thought this might be memory related, but even smaller queries that run fine via Hue interface are generating this error. It is very intermittent. Any help leading me in the right direction to troubleshoot would be very helpful.
... View more