I'm putting this out there mostly to get it indexed, I don't currently have a problem to fix.
Setting up a new HDP cluster, I ran into a problem where hive shell crashes after running a query.
hive> select * from my_table limit 2;
[2 rows of valid output]
hive> Exception in thread "main" java.io.IOException: Bad file descriptor
[Java stack trace]
[me@n01 import]$ _
Looking in /tmp/me/hive.log, I found this message among the wreckage:
2017-11-16 17:18:40,278 WARN [main]: hdfs.BlockReaderFactory (BlockReaderFactory.java:createShortCircuitReplicaInfo(506)) - BlockReaderFactory(fileName=/apps/hive/warehouse/some.db/my_table/000000_0, block=BP-1039108363-10.96.99.32-1510818878096:blk_1073741949_1125): I/O error requesting file descriptors. Disabling domain socket DomainSocket(fd=512,path=/var/lib/hadoop-hdfs/dn_socket)
java.io.IOException: size of shared memory segment was 0, but that is not enough to hold even one slot.
Relevant or not, it did give me the idea to hunt down and disable the option HDFS Short-circuit read in Ambari's HDFS config. Hive shell no longer crashes after this change.
I have no idea if this is a 'correct' fix or not, but I thought I'd put it out there for the next person who has similar problems. If someone has a better idea of what could be wrong or what should be fixed, I'm all ears.
HDP-220.127.116.11 on CentOS 7, using MySQL 5.7 for any task requiring a relational store.