Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Jar files created when invalidate metadata is ran on node running catalog server

SOLVED Go to solution

Jar files created when invalidate metadata is ran on node running catalog server

Explorer

Hi

Has anyone faced issue with impala catalog server creating jar files and storing it in /tmp dir on the node where impala catalog server is running? Like below.

Everytime i run invalidate metadata on any database, it creates 2 jar files like below and doesnt delete those after it finishes.

These files keep on increarsing and overtime start causing below issue

Query: invalidate metadata
ERROR:
FSError: java.io.IOException: No space left on device
CAUSED BY: IOException: No space left on device

 

Also resulting in failure of catalog server and impala daemon eventually.

This happens with only one of my cluster. All my clusters are running on same version. No change done recently.

Using CDH 5.8.3, Impala Shell v2.6.0-cdh5.8.3.

 

Jar files created on running invalidate metadata command.

-rw-r--r-- 1 impala impala 68246195 Apr 4 12:54 0079d271-f044-46be-9580-7d98cd4fced2.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:53 1dd950d5-2db9-4a88-9043-02dff869697c.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:53 2c910b59-83fc-4cde-96d3-3ac086f9fcb2.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 13:02 318ef680-2ea2-47fb-b688-8df439af676c.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:50 3b15f36a-5353-4553-bee6-96c5ac807703.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:53 43e13701-dad8-4892-a19c-43125dbaf1e1.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:50 51a5b806-fb0c-444a-a989-87e832501711.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:55 54ef3cd7-ea8d-4932-a032-9b3f63f5a60b.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:58 81dd054d-d720-4d11-826a-eba2518e4381.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:53 93cc7752-c80c-4f70-a47c-14681fd5c487.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:58 9a25bded-1cd7-4773-8962-0fc41705f728.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:52 a70b6d71-191e-494a-a78e-63a03642e0e9.jar
-rw-r--r-- 1 impala impala 68246195 Apr 4 12:54 a7a102f8-9abe-4388-90a4-8abd85bb9c09.jar

 

Abhishek
1 ACCEPTED SOLUTION

Accepted Solutions

Re: Jar files created when invalidate metadata is ran on node running catalog server

Master Collaborator

You are correct, sorry for the trouble.

 

These known issues have been resolved as part of:

https://issues.apache.org/jira/browse/IMPALA-3983

https://issues.apache.org/jira/browse/IMPALA-3974

 

 

4 REPLIES 4

Re: Jar files created when invalidate metadata is ran on node running catalog server

Master Collaborator

You are correct, sorry for the trouble.

 

These known issues have been resolved as part of:

https://issues.apache.org/jira/browse/IMPALA-3983

https://issues.apache.org/jira/browse/IMPALA-3974

 

 

Highlighted

Re: Jar files created when invalidate metadata is ran on node running catalog server

Expert Contributor
this is a huge bug cause my CDH down. my CDH version is 5.10.2, and impala version is seems 2.7. but i have checked the your post link, this bug has been fixed in impala 2.8. could you give me advises how to resolve this issue except upgrade CDH? i am not sure CDH can upgrade impala alone or not ? thanks.

Re: Jar files created when invalidate metadata is ran on node running catalog server

Expert Contributor
could you tell me how to resolve this impala catalogd issue of CDH 5.10.2 ?

thanks.

Re: Jar files created when invalidate metadata is ran on node running catalog server

Master Collaborator

CDH5.10.2 should have the fix for that specific issue.