Reply
Highlighted
New Contributor
Posts: 4
Registered: ‎09-05-2016
Accepted Solution

Too many open files exception when upsert

Hi,

 

I have several batch processes doing upserts with kudu java API. Whit this scenario I got the exception below a lot of times. It doesn't happen when I try to do it in development mode launching one process at time. Should I configure something to allow multiple accesses to kudu tablet servers? 

 

Caused by: java.io.IOException: Too many open files
at sun.nio.ch.IOUtil.makePipe(Native Method)
at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:65)
at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
at java.nio.channels.Selector.open(Selector.java:227)
at org.kududb.client.shaded.org.jboss.netty.channel.socket.nio.SelectorUtil.open(SelectorUtil.java:63)
at org.kududb.client.shaded.org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:360)

 

Thanks in advance

Cloudera Employee
Posts: 50
Registered: ‎09-28-2015

Re: Too many open files exception when upsert

Hi,

This error is coming from your client. Perhaps you are doing something like
creating many KuduClient objects instead of creating just one and reusing
it? Each object uses up resources like file descriptors, etc. Do you have
some code you can share?

-Todd
Announcements
Unanswered Topics
No posts to display.